Brainwave actuated apparatus

ABSTRACT

A brainwave actuated apparatus has a brainwave sensor for outputting a brainwave signal, an effector responsive to an input signal, and a controller operatively connected to an output of said brainwave sensor and a control input to said effector. The controller is adapted to determine characteristics of a brainwave signal output by said brainwave sensor and based on said characteristics, derive a control signal to output to said effector.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims benefit and priority from U.S. provisionalpatent application No. 61/351,725, filed on Jun. 4, 2010, the contentsof which are incorporated herein by reference.

BACKGROUND

The present invention relates generally to brainwave controlled devices,programs, interactive environments, and the like. Typically brainwavesare read using electrical contacts to the brain of one or more users.The electrical signals are amplified and supplied to a signal processingdevice.

SUMMARY OF THE INVENTION

In one aspect the present invention includes a brainwave actuatedapparatus with a brainwave sensor for outputting a brainwave signal, aneffector responsive to an input signal, and a controller operativelyconnected to an output of said brainwave sensor and a control input tosaid effector. The controller is adapted to determine characteristics ofa brainwave signal output by said brainwave sensor and based on saidcharacteristics, derive a control signal to output to said effector.

The brainwave sensor could be borne by the body of a user and theapparatus can have an environmental interface for interfacing to theenvironment of the user. The environmental interface can be coupled withthe effector, correlated with the effector, or could itself be theeffector. The effector could be an affective effector that expressesaffect to other persons.

In another aspect, the invention provides a performance improvingmethod, comprising: sensing a brainwave signal; determiningcharacteristics of said brainwave signal; and vibrating a device with anintensity dependent upon said characteristics.

Various other embodiments and aspects of the invention are alsodisclosed.

In various embodiments, a brainwave actuated apparatus is part of anaffective communications device, physical or virtual, or in mediatedreality or cyborgspace.

In some embodiments I provide personal safety devices, personal datacapture systems, medical devices, entrainment, physiotherapy withbiofeedback, brain music concerts, training for the use of violin,cello, etc., using one or more electrodes in a multi electrode cap. Forexample, a skull cap with 16 electrodes is provided with a copper meshthat functions as a Faraday cage to shield the electrodes and also themesh cap is the ground plane for an antenna that provides wirelesscommunications, telemetry, and the like. A portion of the apparatus maybe permanently attached to the body, in some embodiments, withdetachable portions that provide additional functionality such as aseeing aid, where the Brain Computer Interface (BCI) portion of theapparatus may be permanently attached and certain extra task-specificfeatures may be added as desired. For example, in a task-specific seeingaid, such as for certain specific seeing tasks (e.g. high contrast worksuch as electric arc welding) a specific kind of computer vision systemcan be plugged into a BCI cap. The BCI cap or “ThinkingCap”™ is designedto accept various plug-in modules such as for extra senses or extraeffectors (as if, for example, extra body parts, of a sort). Additional“eyes”, “ears”, and the like can be plugged into the ThinkingCap™ asdesired.

Some embodiments function as a seeing aid, and visual memory aid, or asa form of assistsive-assistive technology. The invention can alsofunction as a Personal Safety Device (PSD) like the “black box” flightrecorder of an aircraft, but instead the invention provides a walkrecorder, or personal capture device that captures EEG together with EVG(Electro Visuo Gram) for example.

Some embodiments may include a combination of brain electrodes, surfacemount dry or wet electrodes, implanted or partially implantedelectrodes, DermaPlants™, and the like, together with vibrotactileeffectors. In some embodiments the electrodes are read-only and thevibrotactile effectors form the opposite pathway back to the brain. Inthis way, in some embodiments there is a complete HumanisticIntelligence feedback loop without the need to provide electricalstimulus to the brain, as some persons may find this uncomfortable. Insome embodiments EEG is not needed at all. For example, a skull cap withvibrotactile effectors may read out of from a wearable computer fed by a3d range camera such as a Kinect™ range camera, and write tovibrotactile effectors or electrodes or both. In another embodiment, the3d range camera is responsive (by way of the wearable computer and BCI)to EEG signals in the feedback loop that adjusts the parameters ofvision as then perceived by the vibrotactile effectors.

This apparatus therefore can provide vision to a blind person withoutthe need to electrical stimulate the brain. It can do this bycombination of occipital lobe readout to control 3d camera parametersthat then images onto a vibrotactile effector array on the skull cap.The vibrotactile array moves with the head of the wearer, to spatializethe environment by allowing the wearer to scan the head side-to-side, orthe like, and therefore, with a narrow Field of View (FoV) camera likethe Kinect™, a blind person can find their way in a natural head-centricmanner that mimics eye-based vision. The orbit of the whole head thenreplaces the orbit of the eyeball. With VideoOrbits™ imagestabilization, the 3d environment can be scanned and understood in aholistic way and spatialized as a natural direct-user-interface. Thisallows for a reality-user-interface (visual reality itself as auser-interface).

In some embodiments the system also works with SSVEPs (Steady StateVisually Evoked Potentials), so, for example, a flashing light is shownto a user and the same waveform is read, via lock-in amplifier, or thelike (monitored brainwave entrainment).

In some embodiments, eyeglasses such as EyeTap or virtual realityeyewear includes a headband to keep the glasses from falling off.

The headband goes behind the head to the occipital lobe, and may alsowork with dural electrodes, dermaplants, the ThinkingCap™, or the like.

The eyeglasses may also provide P300 (Positive, 300 milliseconds), andaudiotory p300 with earphones.

Time-locked EEG signals known as event-related potentials (ERP) areuseful in this context.

Wearable, implanted or dermaplanted systems are also possible in thiscontext.

Dry readout through hair is also possible in a flexible eyeglass baseddisplay or the like, as well as in headworn apparatus.

A musical instrument like Spa Hero or Hot Tub Hero presents the playerwith colored lights and asks response on hydraulophone to get a score ina game.

Now it can also work by a person imagining what they're going to play,and the brainwave pickup of intention of what will be played or when.

In the future one may detect a person's intended actions fromevent-related potential (ERP), and with signal averaging of the ChirpletTransform, doing it 100 times or so, we can obtain useful information.

There is provided some sort of virtual on/off switch for the thoughtreading device, such as a thought reading camera that can be turned onand off when desired.

This works by going into a known sequence of brain states, such as statetransition diagram from low Alpha low Beta to Beta high Alpha low, thenAlpha high Beta Low then both high, or the like.

Also there is provided an adaptive system that changes the parameters ofdetection in accordance with changes in user condition, etc.

There is provided detection of intention state versus null state.

There is also provided sensory motor response: imagine moving your armor foot, to get SMR (sensory motor response) data.

There is provided use of various ERPs such as visual N400 (negative 400milliseconds), for use in aVMP (visual memory prosthetic), as well aswith P2 (P200 i.e. positive 200 ms) which is involved in the memoryprocesses, and visual N1.

The eyeglass safety band makes use of the fact that the Alpha waves arestronger on the occipital lobe.

The measurement of P300 is also occipitally, so that lie detection andthe like can work with the thought reading camera of the invention.

Motor signals at top of head are read with a headband that supports theeyewear or the like, giving use of the sensory motor cortex.

There is also provided fabric electrodes for use with a hat, or thelike.

Improvements to previous EEG work include detecting transitions andresponding to transitions. Transitions are most notably signified bychirps (changes in frequency). This provides quicker response bydetection of transitions. As compared with PLL, PLL is poor at low SNR.Chirplet Transform gives better performance at low SNR.

Alpha-Theta transition: Alpha waves bring about creativity, but so doTheta waves [Brain Activity by Terrance A. Bastian].

Much interesting activity rests at the transitions between these variousstates.

The Alpha Theta range arises in self-hypnosis, meditation, and religiousteachings [Bastian].

Beta waves arise from focused concentration and mental calculations, orthe like.

Drugs can be prescribed to help in meditation, e.g. to arrive at certainbrain states, especially to help in problems of stress, but the drugsmay have undesirable side effects, as well as a lasting aftereffect.Thus it may be desirable to control the state by methods such asbiofeedback (i.e. closed-loop entrainment, and the like).

Whereas brainwave control may arise from comparison between Alpha andBeta, (i.e. between relaxation and concentration), for an inexperienceduser, a more experienced user can independently control Alpha and Betato some degree. For example, an experienced user may be able to elicithigh degrees of Alpha and Beta activity simultaneously (i.e. high energycontent in the 8-12 CPS range as well as in the 12-30 CPS range at thesame time).

A system that trains a user, through, for example, biofeedback, may helptrain the user for such skills as may be desired in activities likearchery or golf that require simultaneous concentration and relaxation.

Thus an apparatus of the invention may help people improve theirabilities at such tasks by training with biofeedback.

In these various biofeedback means, a display means is provided in someembodiments. This may include a CRT or LCD or similar computer screen,but alternatively it is desirable that we can get into the relaxed stateby closing the eyes. In this sense, an alternate form of biofeedbackstimulus is one of hearing or feeling. Hearing biofeedback can be, forexample, binaural tones for brainwave entrainment. In 1839 Heinrich Dovediscovered that separate tones in each ear result in perception ofbeats.

Another form of feedback can be the sense of feeling. Many people soakin a hot tub to relax, or seat themselves in a comfortable chair.

The chair may be underwater (as in the fiberglass seat of a hot tub) oron land (as in a comfortable First Class airline seat in an inflightentertainment embodiment of the invention).

In one embodiment, there is provided one or more brainwave controlledpumps that pump fluid such as air or water against or near the user'sbody. The fluid may be heated and the temperature may be controlled bythe brainwaves. Preferably there are a range of frequencies and effectsat a plurality of pumps, such as one for the lower back and one for theupper back, or the like.

Isochronic tones may also be used with periodic pulse trains,quasi-periodic tone-bursts, or other energy hurts such as spreadspectrum (e.g. periodic or somewhat quasi-periodic chirp bursts or thelike).

Tactile and audiovisual entrainment, biofeedback, or the like is used toeffect thalamic stimulation of the cerebral cortex, to affect corticalactivity in a frequency range around 1 to 30 CPS over a large area ofthe body such as by tactuators, seating, hot tub jets, as well asaudiovisual stimulus.

It is known that television has a sort of hypnotic effect on thewatcher, thus causing different brain states to be reached. Similarly, acomputer screen is directed in a more structured way, as part of abiofeedback loop, especially in the context of a relaxation tub or seat,such as an in-flight entertainment or relaxation application.

Another embodiment comprises a dream capture machine to capture dreams.The device is programmed to wake up the user from REM sleep which is thetime when reams are most vivid, and then use non-movement of body tocapture and re-enter. A Dream Editor is provided by way of using EEG toannotate the dream without movement of the body so that the user canre-enter the dream.

Alternatively, a Twiddler is used to enter dream text without movementof the body too much that the dream buffer is cleared so that the dreamcan be re-entered.

The Twiddler system uses the steps of:

-   -   (1) detecting when to awake the user;    -   (2) awakening the user then;    -   (3) accepting input text that describes the dream, said        accepting not requiring appreciable movement of the body;    -   (4) entrainment to re-sleep the user.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic diagram of a mind-and-body embodiment,

FIG. 2 is a schematic diagram of a musical instrument embodiment,

FIG. 3 is a schematic block diagram of a in-flight entertainmentembodiment,

FIG. 4 is a collection of graphs illustrating the Chirplet Transform inthe context of the invention as it applies to tracking evolution ofbrain states, and the like,

FIG. 5 is a side view of a violin bow made in accordance with anembodiment of the invention,

FIG. 6a is a schematic diagram illustrating use of an embodiment of theinvention applied to a toothbrush; in other embodiments, a dentist'sdrill, cleaning implement, massage implement, health aid, or the likemay be substituted for a toothbrush,

FIG. 6b schematically illustrates a multimotor embodiment of thetoothbrush of FIG. 6 a,

FIG. 6c is a schematic diagram illustrating use of an embodiment inwhich an environmental interface is an antenna which is also theeffector, by virtue of the fact that the antenna is also visible toothers (e.g. it is anthropomorphised as tail),

FIG. 6d is a schematic diagram of an acoustic embodiment,

FIG. 6e is a schematic diagram a vehicular embodiment,

FIG. 7 is a schematic diagram a hot tub embodiment,

FIG. 8 is a schematic diagram a ThinkingCap™ embodiment, and

FIG. 9 is a schematic diagram another ThinkingCap™ embodiment that hasportions inside the head and outside the head of a user.

DETAILED DESCRIPTION

FIG. 1 is a diagram outlining an embodiment of the invention using amind and body interface. A user 110 wears an electroencephalographydevice, EEG 120, with a wearable computer system, PROCESSOR 130.

The general idea of fitness to the mind and the body is of universalimportance and has been well recognized throughout the ages. Forexample, the slogan that appears on most letterhead and officialdocuments of the Massachusetts Institute of Technology is “Mans etManus” which is Latin for “Mind and Hand”, or more generally, mind andbody. The basic idea is that we must understand theory at a deep levelof the mind (mans) but also have the capacity to execute the theorythrough real world applications of the theoretical concepts, and takephysical action (manus).

In training for this kind of real world symbiosis between mind and body,user 110 may be seated in a hot tub, or an airline chair for in-flightentertainment, relaxation, or training exercises, or the like, or in aspa or gym facility for entertainment, relaxation, exercise, ortraining.

The hot tub, seating, exercise equipment, or the like, may include anelement for providing variable tactile stimulation. Such an element issometimes referred to herein as a tactor. As used herein, a tactor is atype of transducer which converts an electrical signal to a variabletactile stimulation and which may also be capable of converting atactile stimulation to an electrical signal. The tactor 140 in FIG. 1 isa vibratory chin-up bar which is the bottom rung of a ladder 145 whichacts as a piece of exercise equipment.

FIG. 1 shows three points in time: leftmost is when the user firstapproaches the equipment and grabs the bottommost rung of the chinupladder. At center is depicted when the user 110 begins to do a chinup.Rightmost is depicted when the user 110 does a chinup and a “mindup” atthe same time (i.e. the user levitates himself or herself with his orher mind, while doing a chinup.

In this illustration, the tactor 140 is the chinup bar itself, which is,here illustrated, as the bottommost rung of ladder 145, along with alinear actuator that can raise the rung under computer programmedcontrol by way of CONTROLLER 150. Controller 150 is a microcontroller orcomputer system equipped with interfaces. A satisfactory microcontrolleris an Atmel ATMEGA48 microcontroller, and one of the six PWM (PulseWidth Modulation) outputs is operably connected to a linear actuatorthat moves the ladder rung up and down. A satisfactory linear actuatoris a motor gear and cable system with steel cables inside both sides(left side and right side) of the frame of LADDER 145 made such as toraise one or more rungs of the ladder in response to an input to theprocessor. Input, such as from a spectral response in the range of 12 to30 CPS (Cycles Per Second), corresponding to Beta waves from the brainof user 110, is collected from EEG 120 by processor 130, and transmittedwirelessly to controller 150. Analysis on processor 130, or controller150, or a combination thereof, determines a state of concentration ofuser 110, such that the ladder rung of ladder 145 rises in proportion tothe concentration factor of user 110. User 110 can raise this rung withthe power of thought alone, i.e. simply by thinking in the right way,i.e. concentrating, such as to generate brainwave activity with a highdegree of amplitude in the 12 to 30 CPS range.

User 110 can raise the chinup bar while doing chinups, i.e. experience asynergy of mind and body that allows his or her body to be raised up.This mind and body experience is not mere levitation with the mind, but,rather, a cooperation between his or her own mind and body thataccomplishes a task that results in an exercise of both the mind andbody at the same time, in unison, such as to train for such tasks asmight require mind and body coordination.

Tactor 140 includes a tactuator (tactile actuator) which vibrates thetactor at a selectable frequency. This results in a tactile stimulususeful for brainwave entrainment. Preferably tactuator 160 vibrates in arepetition rate in the 1 to 30 CPS range. The actual frequency ofvibration need not be in that range, but the repetition rate ispreferably in that range. For example, a suitable tactuator for use inexercise equipment or hot tubs or seating, is the Clark Synthesis AQ339geophone or hydrophone sometimes referred to as a “Aquasonic UnderwaterSpeaker”, although it is more of a geophonic or hydrophonic device thana loudspeaker (i.e. it is meant to move solid matter or liquid mattermore so than to move air). In applications where the use is notunderwater, but outdoors in light rain, an AW339 will suffice. Othergaming or home theatre transducers may be used, such as “butt thumpers”or “seat shakers” or the like.

The result is “tactile sound”, i.e. a sensation of sound sent to thehuman body directly in solid matter, rather than through air.

As a result, a user in an airline seat can experience an effect withoutdisturbing other people in nearby seats, because the acoustic impedanceof solid matter is much different than air, thus resulting in largeamounts of energy transfer being possible without much disturbance ofthe air.

In exercise equipment, the “tactile sound” can be felt without too muchdisturbance to other people using adjacent exercise equipment. In a hottub, even a communal hot tub or spa, vibration of one individual's bodycan be achieved without too much disturbance to others, if desired.

It helps to classify transducers according to the state-of-matter inwhich they operate. I like to also enumerate them in the order ofincreasing thermal energy of the state-of-matter in which they operate,as follows:

-   1. solid (“Earth”): geophone;-   2. liquid (“Water”): hydrophone;-   3. gas (“Air”): loudspeaker or microphone;-   4. plasma (“Fire”): ionophone.

These states-of-matter correspond (approximately) with the fourClassical Elements (Earth, Water, Air, and Fire).

Since a tactuator is often a device that vibrates solid matter, it mayeither be a geophone or be thought of as being a geophone.

Referring back to FIG. 1, tactuator 160 vibrates the bar with tonebursts that occur at a rate of 1 to 30 tone bursts per second. Sincemany tactuators have problems delivering high energy below 20 or 30 CPS,each tone burst may be delivered at a frequency such as 200 CPS, towhich the human sense of touch is very sensitive. This frequency rangereduces energy requirements. Thus some relatively modest 200 CPS or 250CPS pulses are delivered to stimulate the sense of touch, and this isdone at a rate of 1 to 30 CPS, in order to do entrainment.

Brainwave entrainment is, in some sense, an inverse tothought-controlled technology. Thought controlled technology involvesthe use of the brain to control something. This might be called“telekinesis” or “psychokinesis” in which the mind directly influences aphysical system.

The inverse is when a physical system directly influences the mind.

For example, during an exercise routine, let us suppose that user 110 ispresented with a stimulus of around 8 CPS. Since this stimulus is in thefrequency range of typical brainwave activity, one goal is to cause thebrainwaves, at least in part, to lock onto that frequency of stimulus.

This form of brainwave entrainment works as follows: Controller 150sends a signal to processor 130 which signal programs the processor 130for receiving brainwave signals from EEG 120 and tuning to exactly samefrequency as the excitation is presented, namely, in this example, 8CPS. This is done by way of a lock-in amplifier, PLL (Phase LockedLoop), MuSIC (Multiple Signal Classifier), MFBLP (Modified ForwardBackward Linear Predictor), or the like, so that there is determined adegree of correlation, or the like. Thus, with the user receiving astimulus (say a vibration) at 8 CPS, the system can output an indication(as a biofeedback signal) of the energy in the user's brainwaves whichare at 8 CPS.

More generally, brainwave entrainment need not be limited to sinusoidalsignals of pure tone, but, may instead comprise spread spectrumexcitation, or other arbitrary periodic or quasi-periodic signals thatcan be worked with the equivalent of a more generalized lock-inamplifier.

A standard lock-in amplifier such as a Stanford Research SR510 lock inamplifier can be used for sinusoidal signal detection. For example, wemight excite the user at a particular frequency and then attempt tocoherently detect the existence of that frequency in the subject'sbrainwaves. However, a better approach is to entrain desired brainwaveactivity more generally, with an arbitrary periodic excitation, and thenmeasure, more generally, the response to this very excitation, withsignal averaging, or the like.

Tactile and audiovisual entrainment, biofeedback, or the like, areconstructed such that thalamic stimulation of the cerebral cortexaffects cortical activity, in a frequency range around 1 to 30 CPS overa large area of the body such as by vibratory elements or othertactuators in seating, pulsating hot tub jets, as well as audiovisualstimulus.

Television can have a sort of hypnotic effect on the watcher, thuscausing different brain states to be reached. Similarly, a computerscreen is directed in a more structured way, as part of a biofeedbackloop, especially in the context of a relaxation tub or seat, such as aninflight entertainment or relaxation application, or exercises for themind and body.

Various forms of SSVEP (Steady State Visual Evoked Potentials aredisplayed on SSVEPTV 170 (Steady State Visual Evoked PotentialTeleVision display). In this way, one or more senses can be stimulatedfor brainwave entrainment while part of an exercise or game or trainingor relaxation regimen is in process.

FIG. 2 is a diagram outlining an embodiment of the invention usingmultiple parallel or sequentially selected tactors, such as vibratoryelements that can provide variable tactile stimulation. In FIG. 2, thetactors are rungs of a ladder 241, 242, 243, 244, etc., but in otherembodiments they could also be handles of various golf clubs, tools,cleaning implements, or the like, or parts of a single object such as anairline seat and seat arms and chair back and tray, which form part ofan in-flight entertainment system or in-flight exercise and relaxationsystem.

The bottommost rung of ladder 145, as depicted in FIG. 2, is fitted witha strain gauge resistance bridge, B0, as well as a geophone G0. The topof bridge B0 is supplied with a greater voltage, marked “+” and thebottom with a lesser voltage marked “−”. The four terminals of eachbridge are connected in reality but FIG. 2 only shows partial connectionin a simplified form for ease of illustration.

The bridges are a matrixed in a 3 by 4 arrangement, to use 3 of the 6analog inputs of the ATMEGA 48. The bridges are supplied by voltage fromoutput pins PB1, PB2, PB3, and PB4 of the ATMEGA 48, as referred to theAtmel ATMEGA 48 datasheet, or the pinout diagram, local cache ofhttp://wearcam.org/ece385/avr/.

Were more tactors present, we simply use more pins, e.g. PB0-7 driving a6 by 8 set of matrixed bridges into all six analog inputs provides 48bridges.

The output of each of the 12 bridges (one for each rung of the ladder145) is shown in FIG. 2 as being connected directly to pins PC0-PC2(refer again to Atmel ATMEGA 48 datasheet for PC0, PC1, PC2, etc.,pinout designators). This is merely for simplicity, because in actualfact we connect the two outputs of each bridge (i.e. left and right) toa differential instrument op amp (operational amplifier) and the outputof that op amp is what is actually connected to the input pins PC0-2.Because of the matrixing, for the 12 rungs, we only require 3 op ampsrather than 12 op amps.

The upper left and lower right resistors in each bridge are actuallystrain gauges on the bottom of the corresponding rung, so that steppingon the rung increases their resistance. The upper right and lower leftresistors in each bridge are strain gauges on the top of thecorresponding rung, so that stepping on the rung decreases theirresistance (i.e. increases their conductivity, thus pulling therightmost output voltage of the bridge more positive and the leftmostoutput of the bridge more negative, such that the differential op ampgives a higher output). Thus stepping on the rung with the foot, orpulling on the ring with the hand, causes a measurable output for eachparticular rung, that indicates flexion. Resistance bridges are in someways analogous to a carbon microphone, and can “hear” sounds and otherdisturbances made in the rungs of the ladder, in addition to slowflexing. Thus the bridges pick up a frequency range that goes all theway down to 0 CPS, i.e. Direct Current (DC). In this sense, the soundspectrum that the bridges “hear” includes the origin, in frequencyspace.

In addition to flexion, we have one or more geophones on each rung thatlisten to vibrations in the rung. Geophones tend to pick up higherfrequencies better, and they can also “listen” and “speak”, i.e. theycan create disturbances when fed with electric input. A suitablegeophone is the previously mentioned Clark Synthesis AQ339 geophone orhydrophone. Alternatively, geophones G0, G1, G2, etc., may bepiezoelectric devices.

HDR Proc. 250 (High Dymamic Range Processor) receives input fromlarge-signal bridges B0, B1, etc., as well as small-signal listenergeophones G0, G1, G2, etc., to obtain extended dynamic range over abroad band of disturbances and thus to sense both subtle and largeflexion or sound or vibration in the rungs or whatever other input isused. The HDR Proc. 250 also outputs to various effectors such as alsothe rungs, or other objects that can be sensed or affected by user 110.

The rungs may be made of various materials such as metals, plastics, orwood. For simplicity, let us consider wood.

A xylophone is a well known musical instrument, and “xylo” is the Greekword for “wood” and “phone” is the Greek word for “sound”. Thus“xylophone” is Greek for “wood sound”, and thus the xylophone is aninstrument made from wooden bars of varying length that are stuck bymallets. The word is sometimes used more generally to describe anapparatus consisting of other materials struck with mallets, althoughthe term “glockenspiel” or “metallophone” is often used when thematerial is metal, and the term “lithophone” is used when the materialis stone, etc.

Since we have an effector and a listener on each rung, we may establisha feedback loop that listens and an effector that actuates, such thatwhen any rung is struck or touched by a user, it resonates at a desiredpitch. Thus we can have each rung stand for any desired musical note,the sound being actually produced by the vibrating wood, such as to bean acoustic instrument, but with pitch defined by processing, such as abandpass filter between listening and feedback to sending.

If we desire, for example, tactor 241, the bottommost rung of theladder, is selected by one foot of user 110, and when flexed, is made toresonate or buzz or vibrate at 220 CPS which corresponds to an A note.The next rung is programmed to vibrate at 246.94 CPS, which correspondsa B note. The third rung is programmed to vibrate at 261.63 whichcorresponds to a C note, and so on.

In this way, the ladder of FIG. 2 is a xylophone, and it is played bystriking, tapping, or flexing the 12 rungs of the ladder, each rungcorresponding to the frequencies as follows:

note; freq/second

A=220.00

B=246.94

C=261.63

D=293.66

E=329.63

F=349.23

G=392.00

a=440.00

b=493.88

c=523.25

d=587.33

e=659.26

As an alternative to the bandpass filter algorithm just explained, wecan also employ (instead of or in addition to, i.e. runningconcurrently) a frequency-shifting algorithm that maps the frequenciespresent to the desired frequency. Thus we shift from DC (0 CPS) up towhatever note is desired, e.g. bottom rung, shift from DC up to 220 CPS,etc. The frequency-shifter may be a simple ring modulator, pitchtransposer, or frequency modulator, or frequency transposer. Frequencyshifting is well known in the art, e.g. when singers are off key, andthe pitch is corrected in post-production or in realtime during a liveperformance. Fourier spectral analysis, synthesis, waveshaping, and thelike, as well as frequency modulation, and the like, are also well knownin the art.

Thus we can make a wooden ladder in which each rung is the same length,but it behaves as if each rung were a differently length tuned to adifferent frequency. In this way, we have a xylophone with infinitesustain. As long as you stand on the bottom rung, you hear an A notesound and it never stops sounding until you take your weight off thebottom rung. When you stand on the third rung you hear middle C and ifyou put more weight on that rung the middle C sounds louder and if youput less weight on it the middle C sounds quieter.

The frequency range depicted here, i.e. 220 CPS to 660 CPS, is tactile,and in fact we may wish to drop the whole bank down an octave so it runs110 CPS to 330 CPS, as that is centered nicely on the most easilyhuman-perceptible range of frequencies.

The aforementioned “bank” is called a “filterbank” or “shifterbank”depending on which of the two algorithms are used. The first algorithmis called “filterbank” and the second is called “shifterbank”, thoughthe frequency shifter, if desired, may be thought of as a form of filteralgorithm.

Thus, in this embodiment, the tactors are both sensors and effectors.

It should be understood that the ladder is just an example, for thetactor 241 can also be an airline seat, or the like, that providestactile stimulation when sat on.

User 110, depicted in FIG. 2, wears EEG 120 apparatus. As he or shesteps on rung 241, the oscillations in the rung may be made to depend onbrain state. The oscillations in the rung may also be made to affectbrain state, i.e. through entrainment, and the degree of thisentrainment may be measured, and itself may be used to influence otheractivity in the system.

As depicted in FIG. 2, the user is flexing more than one rung at thesame time, e.g. rung 241 and rung 249, which sounds and provides tactilefeedback for two notes at once, i.e. low A and high B (“b”=493.88 CPS).More generally the user may flex one rung with each foot and one witheach hand so we may tend to have 1, 2, 3, or 4 rungs in flexion at once.

The tactile action need not take place on a ladder, but could also occuron patio stones, walking, or on various parts of a seat, where a usercould provide a tactile input signal by simply shifting weight from oneside of a chair to another, or leaning back on the seat's uprightportion, or reclining, or resting.

Referring back to FIG. 2, the capacity to programmatically affect thefeedback space the user is in, i.e. “cyborgspace”, means that we canassign dynamically varying meaning to each rung.

For example, the bottom most rung 241 can be made to play the first noteof a song, rather than the first note of a scale. For example, let usconsider the popular children's song “Chim Chim Cheree” from MaryPoppins. The first note of the song is a “C” and the second note is a“G”. Here we have the first rung 241 thus play a “C” and the second rung242 thus play a “G”.

Climbing the ladder thus plays the song, either as notes, or perhapssung, vocalized, by computer, as “Now as the Ladder of Life ‘as beenStrung . . . ” for example. Thus the ladder becomes an andantephone(http://wearcam.org/andantephone/).

The user can move through the song with the body, the mind, or acombination of mind and body, by using entrainment, or thoughtcontrolled technology, to select or be affected by the variousfrequencies on the Ladder as shown.

Every third rung could play a whole chord instead of just a note asfollows, for example, first rung 241 is for a C minor chord: theprocessor selects three passbands, 261.63 CPS, 311.13 CPS, and 392 CPS.These are fed back adaptively to make the third rung resonatesimultaneously at all three frequencies.

The same thing can happen while seated. For example, tapping on the armof a chair, we can get it to resonate at three frequencies at once, toform a C minor chord, by having geophones such as geophone G0 listen tothe tapping, process that information with a bandpass filter orfrequency shifter, or combination thereof, and then output to earphones,or, alternatively, output to the other arm of the chair, or that verysame chair arm that is the input device, or to a combination of thetapped arm and the rest of the chair.

Moreover, various apparatus can be fitted into a seat of a passengercraft to create an interactive experience attractive to first-classpassengers who might be provided this experience free of charge as partof a promotion to upgrade to first class, or might be provided to coachclass passengers for a small additional fee.

FIG. 3 is a diagram outlining an in-flight entertainment embodiment.This illustration is shown from the top, i.e. looking down from theceiling of an airplane or train or bus or yacht or cruise liner or boat,or car, or private jet, or other vehicle or home theatre, or hot tub,pool, spa, or the like, at four seats. Whereas reference is made here toin-flight entertainment, it should be understood that this setup appliesto other settings such as a spa, where users are seated at variousstations, such as might occur in a place like SpaWorld USA in Washingtonwhere there are various stations for users to soak and relax in, such asrows of adjacent seating areas, or also disparate seating areas. Theseats here shown are adjacent, but, by way of networked communications,may be spread throughout the world, e.g. a user of a station in SpaWorldUSA might commune with someone on an airline, in-flight, by way ofwireless networked communications between these different venues. Peopleon cruise ships, yachts, and airlines, and people in hot tubs might allshare in a common collective cyborgspace in a communal experience.

A first user 311 is seated on a chair or airline seat having a leftarmrest with a tactor 341 and a right armrest with a tactor 341R. User311 can tap on one of the armrests, such as monitored and affected bytactor 341. The tactor can vibrate itself but also, each time the tactoris struck or tapped or rubbed by the user's hand, for example, it cancause various events. For example, tapping the tactor 341 can sequencethrough a song, such as an andatephonic song like “Chim Chim Cheree”(Mary Poppins), or “Perpetual Motion” (Suzuki), or any other song thatcan be suitably andantephonized, as described in the ACM article: “Theandantephone: a musical instrument that you play by simply walking”, byS. Mann, which appeared in ACM Multimedia 2006, pp 181-184.

Let us consider the user 311 tapping the arm rest, which gives the notesor harmelody (harmony and melody) according as follows:

Tap 1=“Now” (loud)

Tap 2=“as” (quiet)

Tap 3=“the” (quiet)

Tap 4=“Lad-” (loud)

Tap 5=“-der” (quiet)

Tap 6=“of” (quiet)

Tap 7=“Life” (loud)

Tap 8=“as” or “has” (quiet)

Tap 9=“been” (quiet)

Tap 10=“Strung” (loud)

where every third tap is emphasized because this song is in triple time(i.e. “¾ time” if the unit of each tap is a quarter note).

Songs in quadruple time (i.e. “4/4 time” if the unit of each tap is aquarter note) can be programmed in which case every fourth tap isemphasized, etc. Alternatively other ambient or meditative environmentsare programmed for relaxation, mediation, exercise, or training. Inaddition to tactor 341 there is also a “butt shaker” 342 and a “backshaker” 343. Shakers 342 and 343 respond to controller 350 whichreceives input from (as well as gives output to) tactors 341 and 341R.

Shaker 343 is in chair back 345 and vibrates the vertebrae of the user311 to perform a backrub, massage, or tactile effect of sorts, or issimply to interact and communicate with the user or entrain the user311.

One or more users may play together, e.g. two users can “jam” on thesame song, simply by tapping their fingers to generate the song.

This is not merely tapping to the beat of a song, but, rather, thetapping actually controls the volume of the sound through a frequencyshifting schedule, i.e. a song matrix, that is loaded or reloaded foreach phrase of the song, and each song.

For example, suppose users 311 and 312 decide on a song like “Chim ChimCheree”. Once they agree on a song, the frequency shifting matrix isloaded,

and they can tap out the song to generate it. The controller 350, whichis a computer or microcontroller, or the like, listens to input fromtactors.

The first strike or tap is detected, and the processor selects threepassbands, 261.63 CPS, 311.13 CPS, and 392 CPS, from the first entry ofthe song matrix. Each column of the matrix is a point in time, and eachrow is a note. Thus a column is a list of frequencies, and each tap orstrike moves to the next column. In the second column we have only onefrequency 392.00 CPS, and so on, such as to generate chord, note, note,chord, and so on.

The tapping gets the body in motion, and as with drum beats, it is wellknown that meditation and entrainment result. See for example, “Drumsnot Drugs” by Mikenas, Edward E. Percussive Notes. April 1999.

This drum therapy is combined with biofeedback, through SSVEPTV 370.SSVEPTV 370 also displays musical material of the song, as well ascalming material such as scenery that moves with the music or rhythmthrough various phases, such as visual imagery that moves through thefour seasons while user 311 taps out the music of Vivaldi's “FourSeasons”.

Movement, rhythm, relaxation, and exercise and activity are all combinedin the seating.

Additionally, the scenery is displayed with the correct horizon line asdetermined by accelerometer, inclinometer, and sensor 370A or from theairline's own instruments as received by controller 350 which then knowswhich way the plane or boat or car or the like is angled. In this way,the apparatus alleviates motion sickness because the displayed materialis properly oriented with a synthetic horizon line that matches.

Thus if we see a winter scene, or summer scene, the horizon in eithercase matches reality, so that the visual and vestibular cues are matchedbetween cyborgspace and reality even though users 311 and 312 are insome alternate cyborgspace.

Multi-player games are also possible. In another embodiment, users 311and 312 view SSVEPTVs 370 and 371 where they view content. The contentneed not necessarily be limited to VEPs (Visual Evoked Potentials).

For example, two players may view a task on screen, and their brainstates may be additive, so that they must cooperate in getting theirminds in a similar state to complete a goal. For example, we may have acollective mediation in which both players must mediate into high alpha(relaxation) state, and something is displayed as the sum of the alphawaves of all players. This challenges the idea of one player againstanother, and instead results in collaboration, and trains people tocollaborate.

FIG. 4 is a diagram showing signal processing and display of brainwavedata. At the top of FIG. 4 is shown the Chirplet Transform of thebrainwave data, segmented into meaningful quantities. The ChirpletTransform was invented in the 1980s and first published in the followingreference: S. Mann and S. Haykin, “The Chirplet transform: Ageneralization of Gabor's logon transform”, Proc. Vision Interface 1991,205-212 (3-7 Jun. 1991).

An adaptive neural network for processing time-varying frequencies wasalso presented in 1991: S. Mann and S. Haykin, “The adaptive chirplet:An adaptive wavelet like transform”, Proc. SPIE 36th Intl. Symp. Opticaland Optoelectronic Appl. Sci. Eng. (21-26 Jul. 1991).

The axes of the Chirplet Transform are commonly taken to be beginningfrequency 450, Fbeg/S, as shown on axis 410 and ending frequency 460,Fend/S as shown on axis 411. These are denoted quantimetrically, i.e. asdimensionless quantities that range from 0 to 30 CPS, where the perseconds is incorporated into the axis label to make the quantities alongthe axis itself dimensionless, i.e. 0, 4, 8, 12, etc., on an axis thatis, itself, inverse seconds.

The chirplet transform is displayed as a greyscale or color image, intwo dimensions, with color or light quantity denoting intensity. Forexample, high intensities may be denoted in a bright color like red. Ared blob near the origin 400 denotes brain death, i.e. zero frequency,and hopefully that is not the dominant frequency component. Note ofcourse that there will be some components at various frequencies, so wewon't have merely one single point, but, rather, various points in thistwo dimensional image plane that may be displayed on SSVEPTV 370 or thelike, during diagnostics, training, or the like.

Alternatively some further transformation may be done, i.e. imagescorresponding to chirplet transform states may be used to elicit otherstates.

States are denoted, such as state 420 in which the user is asleep at thebeginning and ending of the analysis period. State 430 indicates theuser has just awoken. Thus when the user awakes, we expect to find astrong chirplet transform component in this region because thebrainwaves went from a theta component being strong at the beginning toan alpha component being strong at the end.

State 422 indicates an Alpha to Alpha transition, i.e. a predominance ofremaining in Alpha, i.e. an unchanging relaxation.

State 470 indicates an entry into sleep state (transition from Alpha toTheta range).

The heavily drawn axes 411 and 412 are of great importance because theydenote the boundary between Alpha and Beta states, i.e. betweenrelaxation and concentration.

Region 480 denotes a downchirp region in which brainwaves transitionfrom Beta to Alpha and thus this region denotes a region in whichconcentration is decreasing.

Region 440 denotes an upchirp region in which brainwaves transition fromAlpha to Beta and thus this region denotes a region in whichconcentration is increasing.

Region 490 denotes a steady-state concentration in which concentrationis unwavering.

Thus we can see that the Chirplet Transform is a suitable detector ofvarious brainwave states, and, especially a detector of brainwave statetransitions.

At the bottom of FIG. 4 is a state transition diagram indicating 4states: state S1 (weak brainwave activity), state S2, strong brainwaveactivity in the Alpha region for 8 to 12 CPS, state S3, strong brainwaveactivity in the Beta region from 12 CPS and up, and state S4, strongbrainwave activity in both Alpha and Beta.

Many tasks, games, and activities such as shooting (e.g. police snipersand anti-terrorist task force work), archery, music (e.g. violin andcello performance holding a bow), golf (e.g. a steady hand on the club),and sailing (e.g. a steady hand on the tiller) require a combination ofhigh concentration and high relaxation.

This may seem counter-intuitive but the high concentration alone willnot work for playing violin or the like, because there is a need to haveboth regions of the brain's spectrum (Alpha and Beta) working together.

For this purpose, an upside-down spectrum display 401 may also behelpful, as it has a unique physical interpretation in which highervalues of the spectrum denote deeper thought, i.e. as we go deeper intoconcentrating and thinking deeply, spectrum plot 404 is indicated, i.e.deep in Beta. Deep concentration gives a spectrum like that shown inplot 405, which is deep in Alpha (i.e. high in Alpha wave energy, i.e.high in spectral energy between 4 CPS and 8 CPS).

Using such an upside down spectral display helps people train themselvesto think deeply and broadly.

The goal, is to “will” through conscious thought and effort, thespectrum to take on a shape like that of plot 406. By concentrating onthe plot, the user tries to deepen AND broaden their thinking, to span abroad range of brainwave activities as deep in thought (i.e. as high inamplitude) as possible.

This corresponds to state S4 in FIG. 4, where Alpha and Beta waves areboth strong.

These raw displays are useful but to make the system more fun tooperate, we can also use, instead, visual imagery and otherinterpretations of the chirplet transform, Fourier transform, wavelettransform, and the like, in various ways as part of a biofeedback-basedin-flight entertainment system, or the like.

Additionally, in a preferred embodiment, a neural network is used toclassify and auto-calibrate to changing conditions, especially inambulatory (e.g. wearable computing and cyborg technology) applications.This may be accomplished as follows:

-   -   Capture brainwaves over a sliding window;    -   Apply LEM (Logon Expectation Maximization);    -   Classify the space spanned by a particular user in a particular        circumstance (e.g. a particular business executive in a        particular seat of a particular aircraft, or the like);    -   Auto-calibrate to this spanned space;    -   Initiate biofeedback in the form of a game or training scenario        such as yachting, archery or golf that might appeal to the        demographic of the user;    -   Re-adjust logons in the Chirplet Transform to classify brain        state;    -   Provide a visual “reward” for success, e.g. a graphical        depiction of a high score or success, when there is a high        transition from Alpha to Beta as depicted in State 430, or an        even higher score for being in State S4 where both Alpha and        Beta are high at he same time.

Other classification schemes can also be used, such as by groupingchirplet, spectral, frequency, sequence, etc., information for adaptingto an individual user, classification, and the like. For example,spectral energy and brain states can result in features distributed in atwo-dimensional or higher space of F_(beg) versus F_(end), or Alphaversus Beta, or the like, and clusters of data can be grouped byalgorithms such as some number “K” of closest neighbors (“KNN”) or byweighted K Nearest Neighbors, (WKNN), and the like.

To the extent that the data is often scattered anisotropically, aSingular Value Decomposition (SVD) is performed, or the data may bere-adjusted by way of a Choleski Factorization. Equivalently, PCA(Principal Components Algorithm) is applied.

Thus the user may be presented with a space having one or moredimensions, such as an image (two dimensions) on the SSVEPTV 370, so asto be able to guide a cursor or other object on the screen using thoughtalone, and to achieve meditation, relaxation, or various exercises incombination with other people, and with various tactors and otherdevices in, on, or around the seating or space.

FIG. 5 is a diagram showing an embodiment of the invention built into aviolin bow. The physics of a violin are well known. See for example, anarticle entitled “Why is the violin so hard to play?” by J. Woodhouseand P. M. Galluzzo Plus Magazine Living Mathematics, Issue 31,http://plus.maths.org/issue31/features/woodhouse/index.html.

Here is a brief quote excerpt from the article:

-   -   When you pluck a note on a guitar string, there isn't very much        that can go wrong. You may not play the right note at the right        time, of course, but a single note will always come out at the        expected pitch, and sounding reasonably musical. When a beginner        tries to play a violin, things are much more difficult. When a        bow is drawn across a string, the result might be a musical note        at the desired pitch, but on the other hand it might be an        undesirable whistle, screech or graunch. This difference stems        from a fundamental distinction between the physics of plucked        and bowed strings.        See also, Fiddler—bowing gestures From: schoondw Apr. 12, 2009,        http://www.youtube.com/user/schoondw. One important thing that        makes a good violinist is an ability to focus and relax at the        same time. Accordingly, a violin bow 500, archery bow, musical        bow, golf club, toothbrush, ski pole, or other implement, such        as bow 500 (but not limited to a bow), is fitted with various        brain interfaces, such as servo 501 that adjust parameters of        the implement.

This invention may be applied to various devices like toothbrushes,dentist's drills, polishing devices, floor polishers, massage devices,electric drills, and reciprocating saws like a Sawzall™ (electrichacksaw often used for demolition).

In the case of a toothbrush, servo 501 may simply be a motor that runsthe bristles, in rotary or reciprocating fashion. In the case of aSawzall™, servo 501 is the main motor that powers the Sawzall™.

In the case of a violin bow, servo 501 turns screw 502 to adjust thetension on horse hair 510. This allows the player to continuously adjustthe tension of the bow under program control. The hair 510 runs from thetop end of bow 500 down to the frog 540, which houses a microcontroller,such as an Atmel ATMEGA AVR. This controller 550 receives input from areceiver that receives brainwaves from user 110 or 310, and there isprovided a detection or estimation of Alpha and Beta activity of thebrainwaves. The brainwaves or control from the brainwaves are receivedby antenna 570.

Most violin bows have a coil of wire around near the pad, to help withgrip, and this coil 560 doubles as an inductive loading coil for theantenna 570.

An algorithm and system is used to vibrate the bow to notify the user110 that he or she is not in the “zone” for optimal performance. Thisalgorithm and system is as follows: Upon detecting shallow thought, suchas a lack of simultaneously high Alpha and Beta activity, a vibrator 541is activated by processor computer algorithm in controller 550.

This vibration can be felt in frog 540 and pad 530 as well as anywhereon the bow.

Using biofeedback, the user concentrates on quieting the buzzing orvibrating of the bow, and this has a simultaneous calming and alertingaffect.

Thus the player is able to stay calm and focused at the same time, byusing biofeedback to “will” the bow to “calm” itself.

The handle of whatever implement is being used may also have some kindof user-interface that can be perceived by the user.

Here it is useful for me to introduce the concept of a transmitientuser-interface and a recipient user-interface, in situations where thereare two or more persons involved.

In the example of the violin user, the violin user is the transmitientand members of an audience are recipients. In an embodiment of theinvention one or more recipients may wear an affective computer of sortsthat reads their sense of the experience. For example, recipients mightwear something that reads their EEG signals and determines if they areannoyed or pleased or the like. When one or more audience members areannoyed, the violin bow may give a small electrical “jolt” to theplayer, not strong enough to cause pain or skin burns, but just a little“tingle” to indicate how the audience is feeling. The thing that givesthe player the “jolt” is something I call a “transmitient interface”. By“transmitient interface” I mean a user-interface intended for thetransmitient.

In the example of the dentist, the recipient is the patient and thetransmitient is the dentist. A transmitient user-interface, in oneaspect of the invention, is a handle on the dentist's drill that heatsup when the patient is feeling pain as read by EEG electrodes, EMGelectrodes, etc. worn by the patient. The transmitient user-interfaceworks as follows:

-   1. A recipient is fitted with pain sensors embodied as electrodes    feeding into a signal processors. Alternatively or additionally,    there are strain gages in the arms of the dental chair that sense    hand clenching. A simple machine learning algorithm may aggregate    these two or more signals using a support vector machine or other    simple classifier or pain signal analyzer;-   2. The pain sensors transmit a pain indication signal to a TUI    (Transmitient User Interface) device, such as the handle of the    dentist's drill. The handle may then emit an audiovisual feedback    signal to the dentist, or it may heat up, or cool down, in response    to varying degrees of pain, or the like.

In the latter case (e.g. when the TUI signal is a heating up or coolingdown) the TUI may be said to be or include an “energy modulator”. Theenergy modulator is a source or sink of energy that can be felt by thedentist. In the case of a heating or cooling that energy is thermalenergy. In this way the dentist (transmitient) can feel the pain causedto the patient (recipient).

The result is a system that mimics how real life works. For example,when I drill a number of holes into a concrete wall, I can feel thedrill heat up after a while. People sometimes even personify the drill,as saying “she's overheating”. Thus if the drill is being overworked, Ican feel “her” pain.

Thus the dentist can feel the patient's temper or patience “heat up”.Therefore when my dentist feels the heat in the TUI handle build up, hewould know when to “back off” a little.

Moreover, in another aspect of the invention, in which electroanalgesiasuch as transcutaneous electrical nerve stimulation (TENS)

is being used, the electrodes that are being used to stimulate thepatient may also be used to sense the patient's degree of pain. In thisway the patient electrodes can even be both a TUI and an RUI (RecipientUser Interface).

FIG. 6a is a diagram showing a toothbrush embodiment of the invention.

More generally, within the scope of the invention, a body-borneimplement or apparatus may be hand-held, worn on the body, implantedpartially on or in the body (as for example a medical device), orpartially implanted and partially worn, partially handheld, or the like.

In a toothbrush embodiment the implement may be handheld.

Toothbrush 600 has a motor 610 which is a drive that rotates orreciprocates brush 621 in response to processor 650. Processor 650 isresponsive to brainwaves of a user 110 of the brush or to brainwaves ofa second user 611 receiving input from the brush.

Here is a situation depicted in FIG. 6 of parent and child, in which theparent and child are both wearing brainwave sensors and the parent andchild both feel the action of the toothbrush.

A game is set forth in which both participants, i.e. user 110 and user611 get into the same brainstate to drive the brush. Making the act ofbrushing teeth into a game will encourage it to happen more often andmore willingly by all participants. Additionally the game can havestate-variables that are saved, so that it might encourage people tobrush more often and keep track of brushing. This peer pressure may alsohelp. For example, if a person forgets to brush, they can be representedas an avatar of a goat, or a rat, to symbolize that their mouth stinkslike a goat or a rat. A rodent breath symbol may thus appear in theironline avatar of an online tooth brushing game. This may help use onlinepeer pressure to encourage tooth brushing.

Additionally an oral camera 620 allows parent and child to see cavitiesand provides an educational element to the brushing experience, as wellas captures images that can be sent to or brought to a dentist or oralhygienist for question and answer meetings, or the like.

A tactor 640 adds to what is felt in addition to sensations resultingfrom the motion of the brush due to the action of motor 610, thusallowing the tactility to be multidimensional. Tactor 640 can, forexample, be a geophone that provides sound that is perceived due to bonecondition through the teeth. The tactor 640 may play music in the samekey as motor 610 so that a nice harmony results that can be felt. Thesong may also last a duration of a proper tooth brushing cycle.Accelereomters in the toothbrush handle also may feedback into thisprocess. For example, music can be synthesized that matches the beat orrhythm detected in the brushing. Thus the music follows the rhythm ofthe brushing. The motor is modulated to capture the root note of thechord being played in the music at a particular time and the tactor 640plays along with this music. Tactor 640 can also sense as well aseffect, e.g. in some embodiments tactor 640 may “listen” to the toothbrushing sounds and determine a rhythm as a form of user-input to theprocess.

A bridge 640B measures flexion and gives feedback. One example feedbackalgorithm is such that the parent user 110 can feel the degree of actionof the brush in the child's mouth. To do this, bridge 640B as read byprocessor 650 indicates action, and action is computed and fed to tactor640 which the parent can feel. This action may clock the music along, orotherwise-affect a gaming situation or online experience.

Additionally, stall forces on motor 610 are computed and this additionalinformation is fed to a combined action algorithm that is affected bythe child's brainwave activity that the parent can feel, so that thereis a mutual awareness of painstate and brainstate to sense action, andaffect.

More generally, an implement may have both a TUI (Transmitient Userinterface) and RUI (Recipient User Interface). For example, the handleof the toothbrush may be or may include the TUI, and the head (“businessend”) of the toothbrush may be or include the RUI. Like the dentist'sdrill, the handle of the toothbrush could heat up when the child'stemper is “heating up” (as sensed by the head of the toothbrush or byadditional apparatus such as EEG headband or the like).

Moreover, the transmitient-recipient pathway need not be hierarchical.For example, both ends of the toothbrush could have affectors (affectiveeffectors), so that each person can feel (e.g. by heat) what the otheris feeling (mood or temper or the like). This embodiment can extend togroups of people. For example, a group of workers can be drillingthrough a concrete sidewalk and each worker can “feel” the mood of theirfellow workers. Additionally they might feel the “pain” of the roaditself if they were to drill too close to a natural gas line, forexample. Each drill is equipped with a TUI so that the workers can feelthe results of their collective action as portrayed on this feedbackpath.

More generally, the recipient need not be physically a person physicallypresent, and may, for example, be a representation of another person atanother location, or may be a computer system of sorts, as might thusembody the concept of Humanistic Intelligence as defined in the book,“Intelligent Image Processing” (Author=Steve Mann, published by JohnWiley and Sons, Nov. 2, 2001, 384 pages).

In any of these devices, there may be multiple effectors, multipletractors, multiple motors, or the like. Even one individual TUI or RUImay contain multi-dimensional motors, tactors, effectors, affectors, orthe like.

In the head of the toothbrush, for example, 2 motors can create complexpatterns due to compound motion. When the motors run at rotational ratesin the ratio of small whole numbers, the sound made by the device ismusical in nature, and the patterns can mimic simple parametric curves,similar in some ways to the Lissajous figures, seen on an oscilloscopethat plots one input against another input, each being a sinusoidalwaveform in the frequency ratio of small whole numbers.

Feeding different spectral bands of EEG into each of just 2 motors cancreate a widely varying compound motion that can remain quiteinteresting (much less monotonous) for long periods of time.

FIG. 6b depicts a system with two motors, motor 610 and “Motor2”,depicted as motor 612. A battery pack or battery box 601B is forreceiving customer-supplied cells 601C such as type LR6 (GD) 1.5 volt AAcells. In a simple embodiment, depicted here is a four-cell compartmentthat may be filled with any number of one or more cells. If a pluralityof cells are inserted they may be connected in parallel and supply a DCto DC converter with 1.5 volts that gets converted to a desired orpossibly regulated operating voltage. Alternatively, it is preferable tohave a switching network, comprising a solid state switch 600SW. Theswitch 600SW can be a form of crossbar switch that receives input fromany number from one to four cells, inserted having any polarity (e.g. itdoes not matter which way the cells are inserted) and the crossbarswitch automatically senses the number of cells and their orientationand connects them in series.

In this example, three cells have been inserted with the sameorientation (all of the positive caps upward) and are automaticallyconnected in series to provide approximately 4.5 volts to DC to DCconverter 600DCDC.

The DC to DC converter 600DCDC supplies five volts, regulated, to aprocessor 600P which controls two “H” bridges, one for each motor 610and 612. For simplicity of the drawing, only the connections to motor610 are shown. The connections from “H” bridge 600H to motor 610 aredepicted as connections 610HC.

Processor 600P drives motors 610 and 612 and can be used thusly tosynthesize various musical effects. For example, motors 610 and 612 canrotate at different rates to create a superposition of musical sounds inthe tooth brushing. Let us consider, for example, that the tactor 640 isplaying a simple melody like “Twinkle Twinkle Little Star” that goessomething like “CCGGaaG”, “rest”, “FFEEDDC”, and so on. We might wishmotors 610 and 612 to synthesize a nice harmony like chords “C”, then“C”, then “F”, then “C”, for example. This might be done by having motor610 produce a “C” and motor 612 produce an “E” initially then an “E”again, then an “F”, and then an “E” again, for example. The motor 610 isdriven by the processor 600P to rotate at, or otherwise produce, acertain rate of rotation or vibration consistent with the number ofcycles per second present in a “C” note, or low “C” note which the userperceives as sound.

Low cost motors often do not reliably rotate at a known rate of rotationwithout expensive feedback control devices. While we may have a shaftencoder or Hall effect sensor, or the like, within the scope of theinvention, feeding back to processor 600P, we prefer, instead, to usethe motor itself as the sensor. Motor sense signals 600M1S are picked upfrom the motor back to processor 600P as balanced differential inputs.Processor 600P senses the voltage spikes across the motor. In the caseof a brush based motor these voltage spikes are considerable and areeasily sensed. Alternatively, or additionally, a current sense signal600CL is sensed as a current loop from a wire wrapped around one of themotor leads back to a differential balanced input to processor 600P.

Once processor 600P can sense the motor's electrical properties (e.g.noise made by the motor or periodic changes in the motor's currentconsumption, or periodic voltage spikes, or the like), it can then enterinto a feedback loop to maintain a desired rate of rotation orvibration, and thus a desired musical note or tone pitch sensation.

With a plurality of motors, various musical, audio, and tactile effectscan be created. Such effects include vibrato (chirping) or tremelo. Forexample, if motor 610 produces a somewhat periodic signal that can beapproximated by a signal S_1 at its fundamental frequency or pitchperiod, as S_1=cos(2 pi f_1 t), and the other motor 612 produces asignal approximated by fundamental signal S_2=cos(2 pi f_2 t), then whatthe user may hear or sense or feel or otherwise perceive may besomething of the fowl S=S_1+S_2.

Since S=2 cos(2 pi f_d/2 t)sin(2 pi f_s/2 t),

where f_d is the difference frequency f_1−f_2 and f_s is the sumfrequency f_1+f_2. In certain frequency ranges, e.g typically less than20 cps (Cycles Per Second) the human perception system will perceivethis signal as a tremelo, e.g. as an amplitude modulation envelopesignal having a beat frequency of f_1−f_2.

Thus what is felt or heard or sensed can be much lower in frequency thaneither f_1 or f_2. This allows for the creation of subsonic percepts.

More generally, the apparatus can therefore create subsonic sensorystimulation from higher pitches (and therefore greater forces fromsmaller and lower cost motors) than might otherwise be possible.

Since brainwaves, and in fact much of human tactile sensation, occur atsubsonic frequencies, there can be strong subsonic stimulation that ismatched to the human physiology in various ways.

It is known that human feeling is very effective around 200 cps, or so,e.g. devices like the Tactaid™ make use of these frequencies. Therefore,in one aspect of the invention disclosed herein, we may wish to havetactors that operate at optimal tactile frequencies around 200 cps yetcreate tactile percepts at much lower frequencies as set forth by thedifference frequency f_d. This may be done by using a plurality (e.g.two) of tactors that are co-located. Two co-located motors, or twovibrators, for example, can therefore provide slowly undulating periodicstimulus that may gently massage gums or the like.

The apparatus is enclosed in a waterproof environmentally protectedenclosure or housing 600W.

This invention is not limited to a toothbrush, and in fact may be usedin any context where users control a vibrating or vibrotactile implementthat they or each other or others can feel. Multiple users can use theirbrainwaves to control a local implement or a remote implement or both,allowing a sense of touch to transcend geographical boundaries.

This invention can also take the form of various robotic apparatus thatis either worn or handheld, and can be vibrotactile or simply visual(decorative fashion, for example). Consider, for example, a Halloweencostume that has a robitic tail. The tail may “wag” or gyrate due toeffectors running from brainwaves. In this sense the tail creates anemotional or affective display. A person can then wag his or her owntail by thoughts alone. Additionally, a person can invite others to waghis or her tail by their brainwaves, through wireless link. Withmultiple frequencies present, the tail waves around in various forms notnecessarily limited to simple Lissajous figures, but also more complexpatterns owing to the mechanical resonances that are possible. Fittingthe tail with mass-spring apparatus of sorts, it can function like aniambic (biambic) keyer and swing around in various ways that are partialto a biofeedback loop shared by the wearer and one or more others.

Humans can therefore, like dogs, express their affective (emotional)states and various appendages and robotic indicators (perhaps even lightsources, illumination patterns, in addition to servos and the like) canbe provided. We know, for example, when not to bother a dog. Like theway that a snake will hiss when not wishing to be disturbed, we can reada dog's internal state through the tail. This is one of the reasons dogowners sometimes cut off the dog's tail, so that the dog will lookmeaner or at least be hard to “read” emotionally.

In this way the apparatus of the invention forms a visual roboticemoticon of sorts.

A mechanical robotic tail can also take on a functional as well asaffective aesthetic form.

For example, the functional form of the mechanical tail might be that ofa communications antenna. The resulting Antail™ product (antenna tailproduct) stands up to attain better wireless communications whenever abetter network connection is required.

This provides multifunctionality, e.g. the Antail is both a functionalnetwork connection as well as a visible indicator to others, that anetwork connection bandwidth is increasing.

An attacker, rapist, or robber stalking a victim might, for example, seethe victim's Antail™ stand up. This action, combined with a direct gazeof the victim, might serve as a deterrent.

It is often said that a victim should “Make eye contact if you are beingfollowed by someone who you think is a potential threat. An attacker maybe less likely to strike if they think you will be able to clearlyidentify them.”, WikiHow, “How to Prevent a Potential Rape”,http://www.wikihow.com/Prevent-a-Potential-Rape

But a murderer may have less such worry because by killing the victimthey have removed the possibility of being identified. Thus an attackermay perpetrate rape and murder or robbery and murder in order to not beidentified.

But if the victim is using the apparatus of the invention, one or morevideo cameras combined with a very obvious network connection mayprovide a degree of deterrence. In this way the perpetrator can see thathe has been seen by one more more cameras and that he has aroused anetwork connection bandwidth increase.

While many perpetrators may not be consciously aware of all of thesethings, they can see that there is some kind of apparatus that isresponsive or appears “alive” in some way, and this in itself may serveas a deterrent in much the same way that a cobra puffs up its head whenit feels it may become the victim of an attack.

In operation this aspect of the invention may use a brainwave sensor onthe occipital lobe that reads visual brain activity, or it may also readoff ECG (heart rate) and motion vibration sensors or the like.

When a victim is attacked and shaken the antenna goes up more. When avictim's heart rate goes up, apart from exertion, the antenna goes up.For example, the antenna can go up in response to a saliency index, suchas heart rate divided by footstep rate. When an attacker pulls a gun ona victim, their heart may skip a beat but ultimately, on average, itwill tend to go up over a short time interval, while the footsteps stopor slow down at the request of the attacker. If a person suddenly“freezes” while their heart rate escalates, it might be that they arebeing attacked, and the antenna may thus rise to full transmit capacity.

In some embodiments the antenna or associated personal safety apparatusmay be combined with other deterrents such as pepper spray thatdischarges when or if the apparatus is grabbed. Alternatively theantenna itself may transmit a strong signal in an emergency and this maybe enough to cause RF (Radio Frequency) burns on the hand of an attackerwho tries to grab it.

The Internet connection will always work, generally, but it will rise tomaximal capacity when it is is “aroused”. Thus the Internet itself maybe regarded as a peripheral part of the body that rises and falls inresponse to the mind and body of the person upon which it is borne.

In addition to the prevention or deterrence of physical attacks, theinvention may also be used in a fun and playful way. For example, thevideo cameras and transmitting antenna or network connection antenna canform a visual indicator of some otherwise invisible mental brain state.Such a device might take the form of a CAM™ (Cortical Activity Monitor)or MICA™ (Mechanical Indicator of Corital Activity) that displays brainactivity in a fun and playful way.

Other examples of visual indicators can include robotic whiskers,robotic snake scales on a shirt or other garment, or robotic hair pieceslike a “medusa” wig comprised of hundreds of robotic snakes. Variousshape-memory alloys, electroluminescent materials, and the like, may beused within the scope of this invention.

Additionally, auto darkening eyeglasses may be used to convey a “don'tbother me” affect. Auto darkening screens in eyewear or other vehiclesfor communication such as cars, boats, and buildings, can be used aswell.

Likewise a mediated reality environment can convey the affect and thesensing can be done therein, or one or both can be brought into avirtual environment.

FIG. 6c depicts an Antail™ embodiment of the invention. An antenna tailis moved by a motor 610. The Antail 610A initially rests in a downwardposition of less efficacy and less visibility. When the networkconnection is aroused the Antail stands erect in position of greaterefficacy and greater visibility which is shown in the drawing FIG. 6c asAntail 610E.

The motor 610 is responsive to an output from processor 130. Processor130 also attains its network connection through Antail 610A. Processor130 is responsive to an input from an input device such as EEG120 or acamera 120C.

When user 110 is threatened her Antail stands up to deter would-beattackers. The fact that the Antail is controlled by EEG which is aninherently unreliable medium, only helps to add some deniability. Whenasked “did you stand your tail up at me?” she can say “it has a mind ofits own”. In this sense the tail can be presented as a playful emoticonthat has a “mind of its own” (e.g. is unreliable) so that it is not seenas an overt act of aggression or arousal.

In a more playful sense, when the EEG 120 pickups up on a high degree ofvisual saliency, processor 130 then requests more visual bandwidth toarchive images from camera 120C faster. This need for bandwidth causesan erection of Antail 610A to position Antail 610E which happenswhenever interesting visual subject matter is present. Thus if user 110sees a Person of Interest, she is aroused and her antenna stands up, butthis may happen often accidentally enough to become a cute conversationpiece. For example, the user 110 might have seen something else in hervisual field of view at the same time as another person walked by. Thisrandomness gives the apparatus some useful deniability and renders itlike the fool of the King's court who can be excused for randomutterances, and can therefore speak truth without risking offendinganyone.

Indeed, due to the number of incidents in which police themselves arethe assailants (e.g. police seizing or smashing journalist's cameras, orthe like), the apparatus can provide a safe defense against attack bypolice who would otherwise be offended (and may become violent) by anovert act of documentary practice.

More generally, the apparatus of the invention, in this embodiment,includes an affect indicator (e.g. like the tail that can fall limpduring a mode of less affect and stand erect during a mode of higheraffect), an affect sensor (e.g. the EEG headset or brainwave electrodes,or the like), and a processor responsive to an output from the affectsensor, with the affect indicator being responsive to an output from theprocessor. Preferably the affect indicator provides at least one otherfunction, and has at least two states: a position of lesser efficacythat corresponds to a position indicative of a lesser affect, and aposition of greater efficacy that corresponds to a position of greateraffect.

In another embodiment, for example, the affect indicator might be camera120C itself, rather than (or in addition to) Antail 610. The camera inthis embodiment can “stand up” (e.g. raise upwards, for example) and bemore visible to other people while at the same time become moreeffective at taking pictures. When user 110 is visually aroused, forexample, or startled, her camera or cameras stand erect and captureimages more clearly while also expressing this affect to potentialattackers or in a playful sense simply to other people. When she is lessaroused, her cameras droop down into her hairstyle and are partiallyobscured by her hair Perhaps the cameras can still record images, butthey may be less effective in doing so.

This embodiment of the invention may also be used on or in anautomobile, or building, for example. For example, with the invention, acamera on the roof of a car can “look” at an annoying driver next toyou, and capture video of the annoying or dangerous driver in a way thatexpresses affect to the driver of the other car.

In other embodiments of the invention, user 110 has an affect indicatorthat exists purely (or additionally) in cyberspace or cyborgspace orsome kind of mediated reality. When persons view her through theircameraphones they can see her emotional state indicated by these onlineaffect indicators. Thus a sort of game or conversation piece or informalkind of interaction can take place in an online world.

Participants in the online world can choose to see or be seen in a senseof affect expression that does not require they wear any specialapparatus other than perhaps some simple kind of sensor such as thebrainwave sensor of EEG 120 or the like.

For example, another participant 6200 wearing special eyeglasses 620Ecan see a virtual rendition of a tail on user 110 rise and fall inaccordance with some parameter such as visual salience or arousal of thevisual cortex, or certain other brainwave or cardiographic orrespiratory response, or the like.

The motorized or vibrotactile embodiments of the invention can be usedin various ways for various purposes. For example, physiotherapyapplications might include a massage chair that uses collaboration toallow couples to relax together and massage each other (e.g. rub eachother's backs in cyberspace, across disparate geographical boundaries).

When the frequencies of multiple tactuators in this invention operate inthe ratio of small whole numbers, a nice musical sound results. Moregenerally, some form of acoustic feedback is useful. Musical sounds thatplay in the same key as the motor vibrations are also useful. In oneembodiment of the invention there is a tactuator that generates songsthat can be heard through bone conduction while brushing teeth. In otherembodiments of the invention there are separate sounders that may alsoplay in key with the motor vibrations, and the motor vibrations movethrough a song as the root note of each chord in the song.

In other embodiments multiple motors are used to synthesize sound wavesby superposition. In a preferred embodiment one motor can play the rootnote of each chord while a separate sounding device, speaker, geophone,or the like plays the melody. In other embodiments, one or more motorsplay bass notes while a separate speaker or the like plays other musicalparts of an arrangement that is responsive to brainwaves.

In this embodiment of the invention the TUI and RUI may bespace-division multiplexed, e.g. they may comprise a heating elementarray that has some elements detectable by the transmitient and someelements detectable by the recipient. Alternatively the TUI and RUI maybe one-in-the same. For example, the implement may have a single heatingelement that is detectable by the patient and the dentist or by theparent and the child. In this way the patient can feel the TUI signalwhich is the same as the RUI signal.

There are merits in simplicity of the TUI being the RUI, but there arealso certain aspects of some embodiments of the invention where theflexibility of having the TUI and RUI separate make sense.

In the foregoing the TUI and RUI of the invention were presented asspatially separated, when separated. But the separation, if desired, canbe spatial or temporal, for example.

For example, the invention can take the form of a playfully fun andsilly hat with a directional speakermic (speaker and microphone) shapedlike a dish antenna. The speakermic is on a swivel with servos tocontrol azimuth and elevation. Acting as a microphone it is theenvironmental sensor. Thus it can “listen” to various sounds in theenvironment. As it senses various sounds in the environment it suppliesthis sensed data to the processor 650 which also monitors EEG 120 ofuser 110 to determine whether or not the user is annoyed or pleased orinterested or disinterested, or the like.

The speakermic then “responds” to the sensed sound. For example, ifprocessor 650 determines that the user is annoyed, an inference enginealgorithm running in the processor attempts to correlate this annoyancewith the sensed sound. The sensed sound might have a particular rhythmor periodicity or other temporally-varying attribute that makes iteasily correlatable. An annoyance correlator then makes an inference asto the degree of cause the particular sound is making to the annoyanceof the user. An annoyance attribution table is constructed in processor650 to relate the annoyance to each sensed sound and direction ofarrival of the sound, and the like. A support vector machine or othersimple learning algorithm maps out the space by way of a spatialannoyance map, so as to define annoying sounds by their direction ofarrival, or the like.

Suppose, for example, the annoying sound is that of a poorly-playedguitar. Guitars are commonly sold at grocery stores and departmentstores. For example, Walmart stores often have several different kindsof guitars for sale as impulse-buy items right at the checkout counter.One store had seven different kinds of guitars right at the cashregisters, among the items placed there for people to buy as last-minutepurchases. Accordingly a number of non-musicians buy these extremelow-cost guitars and begin to self-teach and play in public, etc., whilelearning (e.g. “strum, plunk, thump, clunk . . . ”).

Let us suppose, for example, that the user of the invention encounterssuch an annoying sound that has a steady easily-detectable rhythm thatcan be measured using an annoyance rhythm correlator. Periodic patternscan be linked to brain state or brainwave activity by known evokedpotentials detection algorithms. Thus periodic disturbances can becorrelated through signal averaging as well as more modern techniqueswell known in the art.

Suppose therefore that the “ear” (speakermic) swivels around on itsturret and points to find the annoying guitar sound. Processor 650thusly identifies the sound as annoying to the user, and an affectsgenerator is actuated. An affects generator is an effects generator thatis aimed at expressing affect, such as this annoyance. Processor 650reads the annoying guitar sound, and then accentuates the aspects of itthat are annoying. For example, the affects generator frequency-shiftsthe sound to make it a little bit out-of-tune. The processor waits for apause in the sound, and then the speakermic momentarily becomes aspeaker, and plays back an annoyance-processed version of the sound toconvey to the guitar player the annoyance that the sound has caused. Theresulting annoyance burst is, for example, a brief discordant blast ofsampled guitar sound, a little out-of-tune.

The algorithm running on the processor 650 that generates this annoyingsound is herein called an annoyance burst generator or an annoyanceaffects generator. It generates a burst of annoyance based activity andin this example, it blasts that out during a brief pause in the guitarsound source.

In other embodiments the environmental sensor and affector may beshoulder-mounted, e.g. 2 speakermics, one on each epaulet of a garment,or various earlike devices that rise and point to a sound source toexpress to others that they have been heard. The affect generator may bethe mere sight of the ear or ears turning toward the sound, or it may bean annoyance burst or other signal sent back to the sound source. Theaffect generator may be separate from the environmental sensor, or itmay be the environment sensor. For example, a separate speaker andmicrophone may be used, and only one of them need be affectivelydirectional. By affectively directional I mean that its direction isvisible to others, and thusly this directional “gaze” is its way ofexpressing the affect, or is, in fact the affect generator.

In situations where it produces sound or other stimulus, that stimulusmay itself be the affector. Thus the affects generator may be itsoutput, such that it need not have a visible directionality. There is aneed for music sharing in various new ways, as we often see coupleswalking along the street where each person is wearing one of theearphones of a stereo music player. Miniaturization has taken away thespeakers of the old “transistor radio” that previously allowed people tolisten to the same music. Thus affective music sharing andsynchronization is beneficial for all. The invention can for example,pickup music from others who have subscribed to a music sharing world,and create a virtual world in which the affector is a sharing of themusic.

FIG. 6d depicts an apparatus in which the TUI and RUI are temporallyseparable, if desired, and in which the affector is sound output ratherthan a visible directionality. A processor 650 has an array 650A ofsensors such as, for example, a microphone array which can be steered tolisten in various directions by way of directionality 650D. Thisdirectionality allows it to receive sound from a noise source such asnoise source 650N. The upper half of FIG. 6d illustrates microphonearray 650A operating as an environmental sensor at, at some time such ast_1, said time denoted as time 650T1. The lower half of FIG. 6dillustrates array 650A acting as an affector to produce sound such as anannoyance burst, at time t_2, denoted time 650T2.

The invention is not limited to annoyance. For example, sounds that Ifind pleasing may be “reflected” back to their environment in asupportive way. If I hear a song I like, the parts I like are sent back.When someone near me sings a song that brings pleasure to my heart, Ireflect back to it with melodious and harmonious responses akin to“jamming”. In this way if I'm not paying attention my WearComp (wearablecomputer) responds on my behalf and my WearComp “sings along” with thesounds around me that please me.

FIG. 6e denotes an automotive embodiment, but the invention may ofcourse also take the form of smart clothes, smart buildings, smartboats, airplanes, or other body covering. The clothes or car or boat orthe like is just then an extension of the user's own mind and body.

Ideally the entire surface of the automobile or other body covering isan affector display and environmental sensor camera or the like.

Practically, however, such a complete mediated reality would not be easyto attain, unless at high budget such as to make an invisibility suit orinvisibility shield that would be entirely affector and sensor.

More likely we might have a few mediation zones such as affector 690Awhich is a projection screen on the outside of a vehicle 690. Forexample the vehicle or other vessel may be white and the screenssculpted into it to match so that when not affecting, the vehicle is allwhite, and when affecting, the affector appears to hover within thiswhitespace.

An environmental sensor 690 s may take the form of a camera that can“see” (that has a field of view of) that which is beyond the vehicle690. The vehicle may be an outfit for example (e.g. clothes) withdisplay as affector 690A and camera as sensor 690 s.

The vehicle might form a reflectionist rendition of others, as per thearticle ““Reflectionism” and “Diffusionism”: New Tactics forDeconstructing the Video Surveillance Superhighway” by author SteveMann, published in Leonardo, Volume 31, Issue 2/April 1998, pp 93-102.

For example, the vehicle might display the face of a person looking atit. Let us suppose, for example, that someone is getting to close to meand thus invading my personal space. My processor 650 receives inputfrom environmental sensor 690 s as well as EEG sensors or other affector physiological sensors on my own body. A classification algorithm thuscorrelates my own physiological response with this data and when thereis a strong correlation, produces affects. The garment or other vehicle690 around me thus, for example, generates a picture of the person seenon camera 690 s and displays it to that person by affector 690A.

Thus a potential assailant invading the body space or corporeal envelopeof the user is presented with an image of himself, much like thepotential shoplifter sees his own image on the TV screens often placedat department store entrances to remind shoppers that they are undersurveillance.

In this example, my invention reminds potential perpetrators that theyare under surveillance.

The invention can be used in garments or in cars. For example, a personinvading the personal space of a motorist can be presented with an imageof their own vehicle reflected on screen 690S. In this way a personfollowing too closely will see their own car, and can see that theirlicense plate is visible, on the camera-displayed image that might belabeled with “image captured and transmitting . . . ”. Face recognitionor optical character recognition can display the name of the person ortheir license plate number for example. Thus when someone stands tooclose to me they will see a message like “John, don't stand so close tome” displayed on display affector 690A. Optical character recognition ofthe license plate with reverse lookup of the name corresponding theretocan also be used, e.g. “A car registered John Doe is following me tooclosely and invading my personal space bubble”.

A camera dome fitted on the top of vehicle 690 may also be used. Forexample, if vehicle 690 is an outfit, the top may be a hat. If vehicle690 is a car, the top may be the roof of a car. A magnetic mount domecamera with visible turret may be placed thereupon and it turns and“looks” at drivers who are driving dangerously. In some embodiments itis anthropomorphised so as to resemble a human face or eye, and it maywink, blink, or “scowl” at others.

Thus the invention functions as a Personal Safety Device (PSD) to deterstalking, intimidation, invasion of personal space, dangerous driving,and the like.

But the invention is not limited to negative affect. It may also expresspositive affect, for example, visually, as with the previous audio basedembodiment. A visual situation that brings pleasure is reflected backusing a pleasing image derived from the source of the pleasantness.

For example, the vehicle 690 might be a shirt having lights in it thatare an affects generator to message to someone in the environment. Ifthe person standing right in front of my pleases my heart, my heartmight throb through the lights (lights display my ECG) to let thatperson know they've brought pleasure to my heart.

FIG. 7 is a diagram showing a hot tub embodiment. In a hot tub 710,water jets 720 are supplied by a pump 720P through a hydraulophone 720Hor other similar water feature. Processor 550 (not shown) drives pump720P in response to a degree of concentration, relaxation, or the like,while a user 110 (not shown) sits in the tub and is presented with acorporate logo such as logo 730 “WATER MATTERS™” reflected in thewaterline, or a corporate logo “MATERLINE™” at the waterline.

As interesting optical properties of the waterline are controlled bybrainwaves, the brainwaves make water waves, by way of pump 721P andmanifold 721M, the user is entrained to concentrate on these effects, inmuch the same brainstate induced by television.

The desired brainstate is entrained while the user concentrates on thelogos, etc.

In another embodiment, the hydraulophone jets 720 rise and fall withbrainstate, and illumination in the jets is responsive to brainstate.

A multi-user game is also possible. For example, users engage in acollective stream of deconsciousness, relaxation, meditation, or thelike. In one embodiment, users concentrate together to raise the waterjets. An alpha wave consolidator functions like a soft “and” gate(Boolean logic) or a summation, so that when everyone is in a meditativestate, the jets rise, creating the feeling of collective consciousness.

The hot tub can also use multiple pump motors to introduce aspects ofthe invention that are musical or tactile in various ways previouslydescribed. For example, multiple motors can be used to synthesize beatfrequencies that create a throbbing effect or beat effect to follow thebeat of a musical melody or to follow the natural rhythms of the mindand body of users or participants in or around the tub.

The hot tub can also have addressable arrays of affectors such as anarray of energy modulator jets that can vary their temperature,pressure, and other attributes to convey an affect to a spa participant.Multiple spa participants in the same tub, or in different tubs atdifferent geographical locations, can share the waters. For example, anarray of jets can be a TUI to one person and an RUI to another, while atthe same time being sensors by way of being hydraulophonic or havingsome kind of sensory capability such as a fluid user interface asoutlined in U.S. Pat. No. 7,551,161, “Fluid user interface such asimmersive multimediator or input/output device with one or more sprayjets”, by Mann, filed 2005 Dec. 14 with priority document ofcorresponding Canadian Patent 2499784 “WET USER INTERFACE OR LIQUID USERINTERFACE WITH ONE OR MORE SPRAY JETS OR BODIES OF WATER”, filedDecember 2004.

Other sensors can be used. For example a satisfactory RUI is a heartmonitor. The processor reads from the heart monitor and throbs the waterjets in response to the heart. Two bathers at opposite ends of the sametub, or in different tubs possibly in different countries can batheheart-to-heart (e.g. where one's heartbeat throbs the other's waterjets). For example, one persons's tub is or contains an affectsgenerator for the other person's sensor, and possibly vice-versa aswell.

In other embodiments, water showers are modulated as Rainwaves™ thatderive from brainwave controlled valves or pumps. Water faucets andshowers thus also fall within the scope of the invention.

Some embodiments of the invention may be handheld. Others are worn.Others are implanted. There are also various combinations possible. Forexample, portions of the apparatus may be permanently attached and otherportions may be lesser attached.

FIG. 8 depicts a ThinkingCap™ having a portion 800 that permanentlyattaches to the skull of a user. A skull cap mesh 810 forms a fine gridfor the head. The hair is shaved off, and after the device is installed,the hair grows through the mesh holes. In the interim or in combination,a hairpiece may be part of the apparatus. Eyeglass frames 820 fowl partof the apparatus. In this way the eyeglasses are held securely in placeby the mesh 810 and the frames 820 and additionally there is anoccipital lobe comfort band 830 that goes around the back of the headand picks up a connection to one or more DermaPlant™ or indwellingelectrodes for the visual cortex. The permanent portion 800 may havevarious other sensors on it. For example, earlobe pickups 821 andnosebridge pickups 822 help to read more data. Earlobe pickups make goodground points or reference points and can pickup where there is verylittle EMG noise, for example. The point where eyeglass frames 820 touchthe mastoid also form electrical contacts. Particular attention is madeto the structure of the eyeglass frames 820, whether simply part of theskull cap, or separate but attached, they spring and touch the side ofthe face, and contain electrode arrays that read out from the side ofthe face. The large number of contact points of numerous electrodes allaround the face read out affective (emotional) state of the user andalso various facial expressions can be read. Nosebridge pickups can alsohelp pickup EOG (Electro Occulo Gram) eye movement, or the like.Occipital lobe box 831 contains and protects occipital lobe readout andinterface devices such as a neural network reading the occipital lobe.It also houses a rear-looking camera as part of the Personal SafetyDevice (PSD). This need not be a high resolution camera but simply aminiature camera that might capture the face of an attacker sneaking upbehind.

This embodiment may also include various body piercings such as nose,head (similar to hairpiece attachments) and the like. In this way it is,at least in part, a permanent fixture for which removal isnon-negotiable, e.g. on the “will-not, may-not, cannot continuum” itfalls to the “cannot” side of this continuum, such as to make compliancewith a perpetrator impossible rather than the victim seeming to beuncooperative.

There are mount points 850 that can accept various task-specificdevices. For example, a WeldView™ HDR (High Dynamic Range) camera may befitted to the side of the headpiece to look out through a welding helmetas a seeing aid for high contrast subject matter such as seen duringelectric arc welding. As a safety precaution the electric weldingapparatus can look for simultaneous high alpha and high beta brainwaveactivity before strating up (e.g. to make sure the user is fit to safelyuse the machine). Upon such safety check and the machine starts, theuser then can see through camera 840 the electric arc of the weldment,and surrounding material as well.

Various different kinds of attachments and different kinds of seeingaids can thus be fitted to a BrainBus™ which is to the brain as the USB(Universal Serial Bus) is to a computer. In this way the user can plugvarious things into their brain. A wearable computer 860 wirelesslyconnects to the BrainBus as well as one or more peripherals like camera840, or the like.

Additionally, the BCI renders objects and devices, such as the electricarc welder, acting as if true extensions of the mind and body, e.g. asif they were body parts of the user.

The permanent portion 800 has various sensors and effectors on it. Theresult of having it always present on the body is that the mind learnshow it works and over time, it becomes useable and like a part of thebody. By constant exposure to some aspects of the BCI and itsbiofeedback or Humanistic Intelligence feedback loops, the user's brainbegins, over time, to subsume the ThinkingCap™ into the prostheticterritory as if it were a third hemisphere of the brain. Then whendevices are plugged into the BrainBus™ of points 850, they are much morereadily learned and used as if true body parts.

In one embodiment of this invention, there is a continuous capture ofpersonal experiences with EEG, so that images, for example, can besearched based on EEG information like visual salience. For example, aperson can record their entire life and then that data can be correlatedto brainwaves. This may, for example, allow lifeglogging (lifelongCyborGLOGGING) data to be searched automatically.

Various forms of data collection such as camera-switching (betweenrear-facing camera in occipital box 831 and forward camera 840, forexample) can be automated.

Various arrays of cameras, microphones, and other sensors are switchedand allocated automatically based on eye gaze, brain state, etc.

Thus the apparatus does simple things like focus where you look, steerthe beam of a microphone array to where you look, etc., while also usingvisual salience and visual arousal information from the occipital lobeto guide this process during capture, as well as to search through thedata later on.

A doctor might, for example, assist a patient by solving a work-relatedstress problem by looking at EEG and correlating it with activity.

Wearable computer 860 may also include ECG (Electro Cardio Gram) hookupso a video Holter monitor is possible that includes EVG and EEG to helpdetermine causes of stress and avert potential heart failure.

A doctor can see what might have caused heart stress, by reviewing thevideo and also the visual arousal EEG information and visual salienceEEG information and correlating that to ECG data such as heart beatarrhythmia.

FIG. 9 depicts a variation of the ThinkingCap 910 in which threeimplantable devices, e.g. internal devices 901 are shown. These devicesremain totally inside the head 900 of the user. The ThinkingCap 910 hasthree external devices 911.

The external devices 911 transmit electrical power to the internaldevices 901 over a very short distance. This power transmission iswireless so that there is no need for wiring emerging from the body andthus passing through the skin of the user. The external devices 911 alsohave a wireless data communications link to the internal devices 901.The ThinkingCap 910 may be permanently attached in some embodiments,semipermanently attached in other embodiments, or may even be removablein other embodiments.

Modifications will be apparent to those skilled in the art and,therefore, the invention is defined in the claims.

What is claimed is:
 1. A brainwave actuated apparatus configured tooperate a multi-user game, comprising: a first brainwave sensor foroutputting a first brainwave signal originating from a first user; asecond brainwave sensor for outputting a second brainwave signaloriginating from a second user; a device operating in accordance with aninput signal; a controller operatively connected to an output of saidfirst brainwave sensor, an output of said second brainwave sensor, and acontrol input of said device, said controller adapted to: determinefirst characteristics of said first brainwave signal; determine secondcharacteristics of said second brainwave signal; based on a summation ofsaid first characteristics and said second characteristics, derive acontrol signal to output to said device; wherein said firstcharacteristics and said second characteristics are intensities andwherein said summation is a sum of said intensities; wherein the deviceconfigures a multi-user game, the first user and second user beingparticipants in the multi-user game, the multi-user game being aco-operative game having a goal condition triggered at least by the sumof said intensities being a defined brainstate; and wherein the devicecomprises at least one vibratory element, the at least one vibratoryelement capable of providing variable tactile feedback to at least oneof the first or the second user upon the goal condition being triggered.2. The apparatus of claim 1 wherein the at least one vibratory elementvibrates with an intensity dependent upon the control signal.
 3. Theapparatus of claim 1 wherein the device vibrates the musical instrument.4. The apparatus of claim 1 wherein said controller comprises an alphawave consolidator that produces the sum of said intensities.
 5. Theapparatus of claim 1 wherein the control signal is based at least partlyon the summation of said first characteristics and said secondcharacteristics being indicative of each of said first user and saidsecond user being in a meditative state.
 6. The apparatus of claim 1wherein the control signal is based at least partly on the summation ofsaid first characteristics and second characteristics being indicativeof coherence of at least one detected brainwave frequency in each of thefirst characteristics and the second characteristics.
 7. The apparatusof claim 1 wherein the device is local to the first user or the seconduser.
 8. The apparatus of claim 1 wherein the device is remote from thefirst user or the second user.
 9. The apparatus of claim 1 wherein thedevice comprises a Transmitient User Interface (TUI).
 10. The apparatusof claim 1 wherein the device comprises a Recipient User Interface(RUI).
 11. The apparatus of claim 1 wherein the device comprises atleast one of a motor, a tactor, and an effector.
 12. The apparatus ofclaim 1 wherein the device comprises a bridge that provides feedbackresponsive to the control signal.
 13. The apparatus of claim 1 furthercomprising a display device for displaying a task related to the goalcondition of the co-operative game.
 14. A method, comprising: sensing afirst brainwave signal originating from a first user; sensing a secondbrainwave signal originating from a second user; determine firstcharacteristics of said first brainwave signal; determine secondcharacteristics of said second brainwave signal; based on a summation ofsaid first characteristics and said second characteristics, deriving acontrol signal to output to a device; and wherein said firstcharacteristics and said second characteristics are intensities andwherein said summation is a sum of said intensities, wherein the deviceconfigures a multi-user game, the first user and second user beingparticipants in the multi-user game, the multi-user game being aco-operative game having a goal condition triggered at least by the sumof said intensities being a defined brainstate, and vibrating at leastone vibratory element of the device for providing variable tactilestimulation to at least one of the first or the second user upon thegoal condition being triggered.
 15. The method of claim 14 wherein thesum of said intensities are produced by a controller comprising an alphawave consolidator.
 16. The method of claim 14 wherein the control signalis based at least partly on the summation of said first characteristicsand said second characteristics being indicative of each of said firstuser and said second user being in a meditative state.
 17. The method ofclaim 14 wherein the control signal is based at least partly on thesummation of said first characteristics and second characteristics beingindicative of coherence of at least one detected brainwave frequency ineach of the first characteristics and the second characteristics. 18.The method of claim 14 further comprising displaying, on a displayscreen, a task related to the goal condition of the co-operative game.